IS THE IMAGERY DEBATE OVER? IF SO, WHAT WAS IT ABOUT?
Zenon Pylyshyn
Source: http://ruccs.rutgers.edu/faculty/pylyshyn-mehler.htm
I prefer to put this opposition slightly differently: Some properties of the image or of the imagined situation are cognitively penetrable by knowledge and beliefs (many of which are held tacitly and cannot be reported on demand, as is the case with knowledge of grammar or of many social conventions).[2] Other properties may be due to intrinsic causes of various sorts, to the architecture of mind. The inherent nature of mental images might be one of the determinants of certain experimental phenomena reported in the literature, but so might the way in which you have learned certain things you know and the way in which you have organized this knowledge (which may have nothing to do with properties of imagery itself). For example, you have learned the alphabet in serial order so in order to tell whether L comes after H you may have to go through the list, and in order to tell whether certain things happen when you fold a certain paper template to make a cube, you may have to go through a sequence of asking what happens after individual folds (as in the study by Shepard & Feng, 1972, in which it was found that when observers used their imagination to judge whether two arrows in a paper template would touch when folded, it took them longer under just those conditions when it would have taken more folds to actually fold the template). Problems are generally solved by the application of a sequence of individual operations so this in itself says nothing special about mental imagery. It’s true that in order to recall how many windows there are in your living room you may have to count them because the numerical fact is not stored as such. But this has nothing to do with the use of imagery per se, any more than that fact that in order to recall the second line of a poem you need to recall the first line, or that in order to tell how many words it has in it you need to recall the line and count them. There are also many reasons why you might observe certain reliable patterns whenever the subjective experience of “seeing with the mind’s eye” occurs. The burden of proof must fall on those who wish to argue in favor of some particular special mechanism to show that it is at least unlikely that the general mechanism, that we know exists because it has to be used in non-imagery contexts, will not do.
The reason that there has been so much talk (by me and others) about the representations underlying mental imagery being propositional is that there are very good reasons for thinking that much of cognition depends on a language of thought (Fodor, 1975; Fodor & Pylyshyn, 1988; Pylyshyn, 1984). For example, propositions, or more correctly, language-like tree-structured symbolic encodings, are the only form of representation that we know that can take advantage of mechanical reasoning mechanisms, such as computers, and they are also the only ones we know that exhibit the properties of compositionality, productivity and systematicity that are essential characteristics of at least human thought (see Fodor & Pylyshyn, 1988). Although that does not entail that mental images are propositions, the propositional proposal serves as the natural null hypothesis against which to compare any proposal for a special for of representation for mental imagery. It’s not that the idea of images having the form of a set of sentences in some mental calculus is a particularly attractive or natural alternative, but it is the only one so far proposed that is not seriously flawed.
Here is the crux of the problem that picture-theories must face if they are to provide full explanatory accounts of the phenomena. They must show that the relevant empirical phenomena, whether it is the increased time it takes to switch attention to more distant places in an image or the increased time it takes to report details from smaller images, follow from the very nature of mental images or of the mechanisms involved in their use. In other words it must be that these phenomena reveal a constraint attributable to the intrinsic nature of the image, to its form or neural implementation, or to the mechanisms that it uses – rather than to some other extrinsic constraint arising from the knowledge that the subject possesses, or from the way this knowledge is structured, or from the subject’s goals or understanding of the task. If, in order to account for the regularities, one has to appeal to something other than the inherent constraints of the imagery system then, however one might like the picture-theory as a description of what is going on in the mind, it will not serve as an explanation. That is because it is the extrinsic factors that are doing the work and they can equally be applied to any form of representation, including one that is propositional. So, for example, if a picture-theory is to explain why it takes longer to switch attention between more distant places in an image one must show that this is required by the imagery mechanism or medium or format –because of its very nature or causal structure (e.g., because of the physical laws that apply). Otherwise the appeal to imagery carries no explanatory weight. Any form of representation can give the same result merely by adding the stipulation that switching attention between representations of more distant places requires more time (during which, for example, one might entertain thoughts of the form “now it is here”, “now it is there” and so on, providing a sequence of thoughts that simulate what might happen if one were looking at a scene). So if you can show empirically that it is unlikely that the properties you observe are due to inherent properties of the image, as opposed to properties of the world envisioned, the reason for preferring the picture-theory would evaporate.
Although this is a simple point it turns out to be one that people have a great deal of difficulty in grasping, so I will try to provide an additional example. Consider the proposal that images need not be literally written on a two-dimensional surface, but rather may be implemented in a functional space such as a matrix data structure in a computer. Notice that physical laws do not apply to a functional space. There is nothing about a matrix data structure that requires that in order to get from one cell to another you have to pass through intervening cells. In the matrix a “more distant cell” is not actually further away so no physical law requires that it take more time: In fact in a computer one can get from any cell to any other cell in constant time. So if we do require that the process pass through certain other cells, then we are appealing to a constraint extrinsic to the nature of the matrix or “functional space”. Of course one might find it natural to assume that in order to go from one cell to another the locus of attention must go through intervening ones. But the intervening cells are not in any relevant sense located between two other cells except by virtue of the fact that we usually picture matrices as two dimensional tables or surfaces. In a computer we can (though we don’t have to – except again by extrinsic stipulation) go from one cell to another by applying a successor function to the coordinates (which are technically just ordered names). Thus we can require that in going from one cell to another we have to step through the cells that fall between the two, where the relation “between” is defined in terms of the ordering of their names. Thus we can ensure that more such cells are visited when the distance being represented is greater. But this requirement does not follow from the intrinsic nature of a matrix data structure, it is an added or extrinsic requirement, and thus could be imposed equally on any form of representation, including a non-pictorial one. All that is required is (1) that there be some way of representing potential (or empty) locations and of identifying them as being “in between,” and (2) that in accessing places in the representation, those marked as “in between” have to be visited in getting from the representation of one place to the representation of another place. As regards requirement (1), it can be met by any form of representation, including a propositional or symbolic one, so long as we have names for places – which is what Cartesian coordinates (or, for that matter, any compressed form of encoding of pictures such as GIF or JPEG) give us.
The test of whether any particular phenomenon is attributable to the intrinsic nature of images or to tacit knowledge is to see whether the observations in question change in a rationally comprehensible way if we change the relevant knowledge, beliefs or goals. Take, for example, the robust finding that the time it takes to switch from examining one place on an image to examining another increases linearly with the distance being imagined, a result consistently interpreted to show that images have metrical properties like distance. One can ask whether this time-distance relation arises from an intrinsic property of an image or from the observers’ understanding that they are to simulate what happens when looking at a particular display. It is clear that observers can scan an image at a particular speed, or they can scan it at a different speed, or they can simply not scan it at all when switching their attention from one place to another. In our own research, we showed that when observers are given a task that requires focusing on distinct places but that does not emphasize imagining getting from one place to another, the scanning phenomenon vanishes (Pylyshyn, 1981). As in the original scanning experiments, the setup always involved focusing on a place on a mental map and then focusing at another place on the map. But in one experiment the ostensible task in focusing on the second place was to judge the direction of the first place from it (by naming a clock direction). In this and other similar tasks[3] there is no effect of image distance on the time to switch attention between places.
I might note in passing that it is not by any means obvious that people do, in fact represent a succession of empty spaces in scanning studies or in any dynamic visualization. We have obtained some preliminary data (Pylyshyn & Cohen, 1999) suggesting that when we imagine continuously scanning a space between two locations we do not actually traverse a succession of intermediate places unless there are visible features at those locations. When there are such features, it appears that we carry out a sequence of time-to-contact computations to selected visible features along the scan path. Thus it may well be that scanning involves computing a series of times between intermediate visible features and simulating the scanning by waiting out the appropriate amount of time for each transition.[4] Note also that while requirement (2) may seem unnatural and unmotivated when applied to a list of sentences, it is exactly as well-motivated, no more and no less, as it is when applied to a matrix or other “functional space”. In both cases the constraint functions as a free empirical parameter, filled in solely to match the data for the particular case. The same is not true, of course, when the space is a real physical space rather than a “functional” space since there is, after all, a physical law relating time, distance and speed, which applies to real space but not to “functional space”. This is why there has been so much interest in finding a real spatial representation of images, a pursuit to which I now turn.
.
We Make it Easy to Succeed
Successwaves, Intl.
Brain Based Accelerated Success Audios
 |